Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Submodules! #312

Merged
merged 1 commit into from Apr 13, 2013
Merged

Submodules! #312

merged 1 commit into from Apr 13, 2013

Conversation

dahlbyk
Copy link
Member

@dahlbyk dahlbyk commented Feb 4, 2013

SubmoduleCollection

  • this[name]
  • Hackish GetEnumerator()
  • Correct GetEnumerator() - need marshaling help
  • Add() - I'd suggest we tackle this with a follow-up PR

Submodule

  • Name, Path, Url
  • FetchRecurseSubmodules, Ignore, Update rules
  • Init(), Reload(), Save(), Sync()
  • RetrieveStatus()
  • Stage()

Index

Tests

  • Works on my machine
  • Real tests

While I'm working on setting up proper tests, I thought I'd get this out for feedback...

var submodule = repo.Submodules[submodulePath];

var statusBefore = submodule.RetrieveStatus();
Assert.Equal(SubmoduleStatus.WorkDirModified, statusBefore & SubmoduleStatus.WorkDirModified);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wouldn't HasFlag() work here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, but it's internal.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are other tests where we could use HasFlag() (or HasAny<>()) too. From a purity standpoint, they really don't belong as part of the public surface area of a Git library, but is that a good enough reason? Or there's the slippery slope of [InternalsVisibleTo], which would allow Core.Epoch and the public classes in Core.Compat to be hidden.

@dahlbyk
Copy link
Member Author

dahlbyk commented Feb 18, 2013

New set of commits based on latest, with proper tests. First three commits could be merged early if they look good - nothing submodule specific.

Next up: immutability.

I'd appreciate some feedback on other scenarios everyone would like to see tests for.

@Yogu
Copy link
Contributor

Yogu commented Feb 26, 2013

I'm using this fork in an evaluation project and it works quite well. I would appreciate it being merged :)

@Yogu
Copy link
Contributor

Yogu commented Feb 26, 2013

However, I've got a question: is cloning repositories recursively already supported? When I clone a repository, and then call Init() for all the submodules, the submodule directories are created, but empty. How do I set up the .git files and the modules directory? What's the stategy to clone recursively?

@dahlbyk
Copy link
Member Author

dahlbyk commented Feb 26, 2013

I would appreciate it being merged :)

@nulltoken How about I strip out the parts related to mutation for now and hit that as a separate PR? Enumeration, lookup and staging seem reasonably solid...

However, I've got a question: is cloning repositories recursively already supported? When I clone a repository, and then call Init() for all the submodules, the submodule directories are created, but empty. How do I set up the .git files and the modules directory? What's the stategy to clone recursively?

I've not tried using Init() yet - @arrbee?

@@ -0,0 +1,169 @@
#!/bin/sh
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please make sure we have the proper permission to accept 3rd party code here (or remove the hooks if they are not necessary?).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll kill them.

Related: libgit2/libgit2@fc6c5b5

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See 49ce35b (which can be cherry-picked standalone) and updated 69da28c

@dahlbyk
Copy link
Member Author

dahlbyk commented Feb 27, 2013

I've broken the existing mutation members out into their own commit, though I've left all the bindings. If everything else looks satisfactory, I propose we close out this PR and continue with mutation and add/init/update in separate PRs.

@dahlbyk
Copy link
Member Author

dahlbyk commented Mar 3, 2013

Another update on latest, with a new set of properties: HeadCommitId, IndexCommitId and WorkDirCommitId (test = dahlbyk@c81f423#L1R48).

Works on my machine, but Travis build is failing for the two tests that expect a null ObjectId. Just pushed a possible fix.

I believe this exposes everything one would need to manually clone/update a Submodule, though that seems like something we should include eventually.

Another question I had while learning the libgit2 APIs a bit better... Is there any way to tell if git_submodule_init needs to be called, other than manually inspecting config? git_submodule_status doesn't differentiate between config in .gitmodules or .git/config, returning GIT_SUBMODULE_STATUS_IN_CONFIG for both. /cc @arrbee

@dahlbyk
Copy link
Member Author

dahlbyk commented Mar 3, 2013

Just pushed a possible fix.

Looks like d082a81 worked; rebased accordingly.

@KindDragon
Copy link
Contributor

I think you should also support FileStatus Submodule, so we can detect submodules in Index.RetrieveStatus() like does git status

@KindDragon
Copy link
Contributor

I found that git_status_foreach doesn't support that :-(

@arrbee
Copy link
Member

arrbee commented Mar 5, 2013

Another question I had while learning the libgit2 APIs a bit better... Is there any way to tell if git_submodule_init needs to be called, other than manually inspecting config? git_submodule_status doesn't differentiate between config in .gitmodules or .git/config, returning GIT_SUBMODULE_STATUS_IN_CONFIG for both. /cc @arrbee

@dahlbyk Sounds like we should change that part of submodule status to make it clear. I know of an issue with that IN_CONFIG status for some corner cases of git_checkout already, so I'd like to change it so that I can fix those checkout cases. As you probably already know, the submodule APIs have not seen that much usage, so we'll probably have to evolve them...

I found that git_status_foreach doesn't support that :-(

@KindDragon I'm not quite sure what you're saying is not supported? If status finds a submodule, it should compare the SHA of the current HEAD of the submodule to the SHA in the HEAD of the current branch of the parent repository. That is the extent of what's supported in the current status implementation.

We can add flags to enable other behaviors if you can help me pin down what is needed. I can imagine a flag to check if any files in the submodule working directory are dirty (which is useful, but unfortunately somewhat expensive). Or a flag to include the submodule in the status results even if the quick is-HEAD-moved check appears unmodified, so that you can call the deeper git_submodule_status() API to check further on demand. I'm not sure exactly what you are looking for, but as I mentioned above, this facet of the library hasn't been exercised that much, so I'm very interested in feedback (and undoubtably, bug reports).

@KindDragon
Copy link
Contributor

Repository.Index.RetrieveStatus() return all items including submodules, but there is no easy way to determine is current StatusEntry submodule.

@KindDragon
Copy link
Contributor

@dahlbyk
How implement git submodule status --cached <submodule local path>? I just want to get commit hash.
Repository.Submodules[<path>].HeadCommitId?

@dahlbyk
Copy link
Member Author

dahlbyk commented Mar 14, 2013

git submodule status --cached <path> would be Submodules[<path>].IndexCommitId
git submodule status <path> is Submodules[<path>].HeadCommitId

@KindDragon
Copy link
Contributor

@nulltoken when you plan merge this pull request? We want submodules API for our project

@nulltoken
Copy link
Member

@KindDragon It's next on my list. Will be done by end of next week.

@RobertMe
Copy link

Hi,

I would like to know what the status is of submodule add. Is this already being worked on? And would it be possible to add (and update/stage) a submodule without cloning it? I'm building some code which automatically generates files and stores them in a repository. This also needs to have some submodules included, but there is no need to have those submodules available at the location where the files are getting generated (as the repo will always be cloned and after the clone you of course need to run the submodule update --init). I guess that would be possible by only setting the submodules HEAD, right? And is this feature available in libgit2 and will this also make it into libgit2sharp?

Anyway, thanks for the great work on the library in general. It's just so easy to create and update a repository with it 😄

case (int)GitErrorCode.NotFound:
case (int)GitErrorCode.Exists:
case (int)GitErrorCode.OrphanedHead:
break;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be more explicit to return null here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Returning a null handle felt weird, but I guess we do it other places.

@nulltoken
Copy link
Member

I concur with @yorah.

Moreover, going down the "snapshot" road, I think I'd prefer to not expose HeadCommitId, IndexId and HeadId as dynamic properties.

public class Submodule : IEquatable<Submodule>
{
private static readonly LambdaEqualityHelper<Submodule> equalityHelper =
new LambdaEqualityHelper<Submodule>(x => x.Name);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nitpick: Two submodules from two different repos, bearing the same name, shouldn't be considered as equal, should they?

If a Submodule turns into a snapshot, maybe should we add the IndexCommitId, HeadCommitId, WorkdirCommitId as contributors to the equality definition.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nitpick: Two submodules from two different repos, bearing the same name, shouldn't be considered as equal, should they?

I suppose not, but I'm not entirely convinced it matters. Can we just document somewhere that "interaction between objects created from different Repository instances could yield unexpected results"?

Another "identity crisis" repro:

using (var repo = Repository.Init(BuildSelfCleaningDirectory().DirectoryPath, true))
using (var repo2 = Repository.Init(BuildSelfCleaningDirectory().DirectoryPath, true))
{
    Assert.NotNull(repo.Head);
    Assert.Equal(repo.Head, repo2.Head);
}

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That said, HeadCommitId can absolutely contribute to equality. Something feels wrong to me about IndexCommitId and WorkdirCommitId contributing to equality, but I can't put a finger on exactly what it is.

@yorah
Copy link
Contributor

yorah commented Mar 26, 2013

@dahlbyk 0662c27 is a spike related to the removal of handle from Submodule. I thought there would have been more changes, but the LazyGroup<T> was actually made to allow this kind of things!

If we don't like the added complexity of having lazy properties, another solution would be to evaluate the properties when we build the Submodule instance.

Another nitpick while I was at it: I added the Rule suffix to Ignore/Update/FetchRecurse properties (I was confused at first, as Update/Ignore looked like names related to actions that we want to perform).

@dahlbyk
Copy link
Member Author

dahlbyk commented Mar 26, 2013

I like the direction you're going, though I don't like the change in semantics for SubmoduleCollection that the index getter will return an invalid Submodule instance given an incorrect name - I think we should still do an initial lookup (perhaps also fetching Path, Url, other properties?) before constructing the Submodule so that we can return null as appropriate.

I'll try to put together a rebound after work.

I added the Rule suffix...

👍

@dahlbyk
Copy link
Member Author

dahlbyk commented Apr 8, 2013

@nulltoken Updated 3cffa1f with mention of the Mono issue, so it can probably be cherry-picked standalone.

@yorah I'm not a huge fan of how uses of git_submodule_lookup are scattered and don't handle null handles consistently... I'm open to ideas. Otherwise, I think this strikes a reasonable balance between reusing handles and avoiding extra work.

Any other feedback?

}
}
}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops, I think this missing newline originates from my spike 😊

@yorah
Copy link
Contributor

yorah commented Apr 8, 2013

Impressive PR, it's great to see Submodules coming to libgit2sharp!! 👏 👏

@dahlbyk
Copy link
Member Author

dahlbyk commented Apr 8, 2013

I'm not a huge fan of how uses of git_submodule_lookup are scattered and don't handle null handles consistently...

How about cf574fd?

@dahlbyk
Copy link
Member Author

dahlbyk commented Apr 9, 2013

Updated on latest. I left 5de5182 on its own for review, but it can be squashed into ac4a5f6 if you like the change.

@yorah
Copy link
Contributor

yorah commented Apr 9, 2013

Introduce SubmoduleCollection.Lookup()

Elegant solution!

@dahlbyk
Copy link
Member Author

dahlbyk commented Apr 10, 2013

Updated to use new Stage() implementation from #343: TryStage() is now called from Index.AddToIndex().

Also new is 9b7252b, used in a new test that exercises staging a submodule along with other paths.

@dahlbyk
Copy link
Member Author

dahlbyk commented Apr 10, 2013

Introduce SubmoduleCollection.Lookup()

Elegant solution!

Thanks! Squashed!

@nulltoken
Copy link
Member

Wow! You did a fantastic job! 👍

I cherry-picked all of them but the last one.

@@ -72,7 +72,7 @@ internal ObjectId TargetId

private GitObject RetrieveTreeEntryTarget()
{
if (!Type.HasAny(new[]{GitObjectType.Tree, GitObjectType.Blob}))
if (!Type.HasAny(new[] { GitObjectType.Tree, GitObjectType.Blob, GitObjectType.Commit }))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree that we should not throw in this case. However something bothers me.

Technically, this will allow the creation (few lines below) of a concrete Commit which won't mean anything in the super-project as it technically doesn't exist in this context.

How about adding a new GitLink type. This type would derive from GitObject and would be returned as such.

Provided we go down this road,I can think of some additional topics to cope with

  • Inspect any public method accepting a GitObject as a parameter and determine how they should behave when being passed a GitLink.
  • Decide if the GitLink type should be internal-only or public

Thoughts?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree that returning a Commit that's not actually a Commit feels a bit strange. We could certainly handle GitObjectType.Commit as a special case here to return a GitLink, but I wonder if it would make more sense to return a new type from libgit2 (e.g. add GIT_OBJ_LINK to git_otype)?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if it would make more sense to return a new type from libgit2 (e.g. add GIT_OBJ_LINK to git_otype)?

I have the feeling that this proposal may lead to some debate.

How about tweaking this now at the LibGit2Sharp level? When/if this leads to a new entry to git_otype in libgit2, this would later bubble up through failing tests.

/cc @carlosmn @arrbee

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have the feeling that this proposal may lead to some debate.

You're probably right. I see that git ls-tree returns commit; diverging probably doesn't make sense.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about adding a new GitLink type. This type would derive from GitObject and would be returned as such.

21696b8

Provided we go down this road,I can think of some additional topics to cope with

  • Inspect any public method accepting a GitObject as a parameter and determine how they should behave when being passed a GitLink.
  • Decide if the GitLink type should be internal-only or public

Please have a look for yourself, but at a glance I didn't see anywhere that we publicly accept GitObject where we have special cases that depend on the type provided, presumably allowing libgit2 to determine which object types are acceptable.

It would be good to support TreeDefinition.Add(string targetTreeEntryPath, GitLink link) eventually.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please have a look for yourself, but at a glance I didn't see anywhere that we publicly accept GitObject where we have special cases that depend on the type provided, presumably allowing libgit2 to determine which object types are acceptable.

You're right. Thanks!

It would be good to support TreeDefinition.Add(string targetTreeEntryPath, GitLink link) eventually.

I strongly agree with you. However; I'm not sure about the API. How would the user create the GitLink in the first place? Beside this, I'm not sure it would be safe to let pick the path by himself.

I wonder if TreeDefinition.Add(Submodule module) would work? Though, I can still think of a corner case where the TreeDefinition would be used to not build a root Tree, but a subTree (which would result in having the path in the .gitmodules file and the path from the root Tree being different)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How would the user create the GitLink in the first place?

Maybe ObjectDatabase.CreateGitLink(ObjectId objectId), similar to existing CreateBlob() and CreateTree()? It would just call the ctor, but the public doesn't need to know that.

I wonder if TreeDefinition.Add(Submodule module) would work?

That wouldn't really align with the other TreeDefinition APIs, IMO. Perhaps TreeEntryDefinition.From(Submodule module), but that's about as far as I think we'd want to let a high-level concept creep into this low-level API.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be continued: #393

@nulltoken nulltoken merged commit 21696b8 into libgit2:vNext Apr 13, 2013
@nulltoken
Copy link
Member

@dahlbyk Merged!

@nulltoken
Copy link
Member

🎱

@dahlbyk dahlbyk deleted the Submodule branch April 14, 2013 04:54
@dahlbyk dahlbyk mentioned this pull request Apr 14, 2013
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

8 participants